The present invention relates generally to a data processing system, and more particularly to a storage system capable of transferring block level I/O data from a primary storage system to a number of secondary storage systems. The data location information, in the form of a file system, is transferred separately to the users of the secondary storage systems.
Today's availability of massive storage systems, and the ready access to such storage systems though the Internet, has created a demand for such applications as, for example, video viewing on a real-time basis for both entertainment and education. Use of the Internet, for example, as the communicative medium allows video files to be transferred for viewing virtually anywhere in the world. However, making even large storage systems available to large numbers of users carries with it certain problems, not the least of which is the degradation in performance that can result from the bottleneck that most likely will arise when many users attempt to access the same file at about the same time.
One solution is to provide multiple data centers at different locations to distribute the access load that can be expected from the users. This would allow multiple users to access (video) files at one of a number locations. A primary location, such as a movie studio in the case of video files (motion pictures) can distribute the file to all data centers.
Video files are, however, usually managed by a file system of an operating system. Applications, such as a video server running on the operating system, will treat a video as a file due to ease of access. Generally, transfers of video files use file level I/O such as the File Transfer Protocol (FTP) to copy a file from one server to another server via a TCP/IP network. This protocol assumes that data is stored on a file system.
However, data transfer methods based on a file suffer from low data transfer rates that result from the often large overhead of a file system. File systems often employ a logical file management system in which logical “volumes” map to predetermined portions of a storage space, usually implemented with a number of disk devices. To read and/or write a file, the file system also reads and writes the meta data of the file, the information describing the file which often identifies where the file is stored on the volume and its related properties such as the file's name, its size, its last accessed time, and so on. The meta data is also stored on the volume, so the meta data operations accompanying a read and/or write operation on a file can be a drain on CPU power and an I/O bandwidth. This is why the transfer rate based on a file transfer is slow.
There are services capable of providing a file distribution service. Akamai and Digital Island are a few examples of such services, and usually use a file-based data transfer method such as described above and, therefore, suffer the problems mentioned. There is also a remote copy method that provides a block level data transfer. This transfer method is described in U.S. Pat. No. 5,742,792. But, the transfer methods used are not structured for use by applications employing file systems. This means even if the data copy was performed, no application or file system can work with the transferred data.